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Abstract: This paper examines the impact of the national qualifications framework on the education and training 
system in Slovenia in the context of the European qualifications framework for lifelong learning and its influence on the design 
of the Slovenian qualifications framework. Although the role of the European Union in educational policy-making has received 
considerable attention, of which national qualifications frameworks are part of, only a few studies have focused on measuring 
the impacts of the European qualifications framework influenced national qualifications frameworks in Europe. By drawing on 
the theoretical concept of policy transfer in the analysis of European/global education policies, the study shows how European 
qualifications framework policy transfer influenced Slovenian qualifications framework development through soft instruments. 
Furthermore, this study examined the extent to which the Slovenian qualifications framework’s objectives have been achieved 
according to key stakeholders (n = 50) using a quantitative research approach. The findings indicate that, unlike objectives 
related to the Slovenian qualifications framework’s reform role, i.e. support for lifelong learning, the objectives related to the 
Slovenian qualifications framework’s communication role, i.e. recognisability, understanding and transparency of qualifications 
and coordination of the qualifications subsystems, are mostly being met from the stakeholders’ perspective. 
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Introduction 


The role of international organisations (lOs) in global and/or European education policy-making has 
received considerable attention in the scientific community (Caspersen and Frelich, 2017; Jakobi, 2009; 
Kleibrink, 2011; Portnoi, 2016), particularly their role in the establishment of the European qualifications 
framework (EQF), a key European policy instrument for lifelong learning (LLL) (Elken, 2015), and the 
national qualifications frameworks (NQFs) whose establishment have also been strongly supported by 
various IOs around the globe (Chakroun, 2010; Raffe, 2013). Nevertheless, there is still much to be 
learned about their impact on national education systems, policies and practices (Portnoi, 2016) and the 
application of the policies from IOs to the national level (Crossley, 2019; Jakobi, 2009). 

To address this research gap, the current study aimed to contribute to the better understanding 
of the impact of the EQF-influenced NQFs in Europe. This study explored the impact of the Slovenian 
qualifications framework (SQF) on the country’s education and training system. We measured the 
impact indirectly by focusing on the views of the stakeholders, whom we named ‘key stakeholders’, i.e. 
members of the professional public who have been, by their occupational position, involved in the SQF’s 
implementation or usage. Furthermore, we looked into the key stakeholders’ views regarding the extent to 
which the SQF objectives were met and investigated their familiarity with the SQF’s benefits and whether 
their views differed regarding the SQF’s usage. 

In what follows, we briefly introduce the role of IOs in fostering policy transfer and the instruments 
used in the EQF policy transfer, elaborate on the issue of measuring the impact of the current SQF, outline 
the methodological approach employed in this work and present the results of empirical study. In the final 
section, we discuss the EQF’s impacts on the SQF and that of SQF on the education and training system. 
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International organisations and instruments used to facilitate transfer 

Due to globalisation, |Os as agenda-setters, such as the European Union (EU), the Organisation 
for Economic Co-operation and Development (OECD) and the United Nations Educational, Scientific 
and Cultural Organisation (UNESCO), among others, play an increasingly crucial role in the formation 
of the global and/or European education policies. They have been identified as ‘central nodes for policy 
diffusion’ (Jakobi, 2012, p. 391) that are able to transfer policies between countries and promote their own 
policies, although their formal competencies are generally limited. Despite the fact that the [Os generate 
different policy outputs and differ in focus and scope, connections and policy convergence among the 
lOs' policies exist (Jakobi, 2009). The case of the NQFs is a good example of such a convergence (Raffe, 
2013). As influential actors framing education and contemporary LLL policies, |Os promote policy transfer 
towards evidence-based educational practices, the measurement of the effectiveness of education and 
the achievement of goals relating to competitiveness and employability (Mikulec, 2017) through new 
instruments and practices of governance based on knowledge and data generation, peer learning, 
benchmarks, indicators, monitoring, evaluation and funding (loannidou, 2014; Lawn and Grek, 2012). 

Therefore, the IOs stimulate a kind of transnational educational policy transfer that encompasses 
ideas, ideology, practices and institutions, involving multiple actors (Crossley, 2019, p. 4). The process 
of policy transfer should be understood as a continuum that can incorporate various influences, from 
voluntary ones to more coercive ones (i.e. policies that are forcibly imposed on countries or those 
that countries must adopt due to political pressure) (Portnoi, 2016). Furthermore, as Jakobi (2009, pp. 
34-36) pointed out, [Os promote policy transfer and influence national policy development through the 
following identifiable soft instruments: (a) discursive dissemination or the establishment of ideas and 
goals for national agendas; (b) standard setting or the conventions and recommendations as well as 
benchmarking, explicit aims and rules to which states should comply; (c) financial means aimed at eliciting 
specific behaviour (establishing programmes or institutions); (d) coordinative functions or the instruments 
of surveillance monitoring progress toward common policy aims; and (e) technical assistance enabling 
States to achieve set policy aims. 


European education policy and instruments used in the EQF policy transfer 

The EU is one of the key agencies contributing to the formation of European education policy. The 
adoption of the Lisbon Strategy in 2000 marked the starting point in establishing a European education 
policy defined by common goals, implementation tools and financial resources, although the EU formal 
competencies in the field of education were limited by the subsidiarity rule (Mikulec, 2017). For the purpose 
of formulating and maintaining the European education policy, the European Commission (EC) adopted 
an open method of coordination (OMC) to improve the effectiveness, coordination and measurability 
of the outcomes of various LLL policies. The OMC established a new form of multilevel educational 
governance that (a) is exercised in the form of ‘soft law’ (e.g. recommendations, guidelines, resolutions, 
conclusions, etc.), (b) is based on soft law voluntary implementation from the member states, (c) and 
strives to establish monitoring mechanisms through benchmarks and indicators in order to measure and 
compare the progress of member states (loannidou, 2014; Lawn and Grek, 2012). 

The EQF is one of the numerous soft law recommendations produced by the EU. Whilst the 
initiatives leading to the development of the EQF can be traced back to 2001 (Elken, 2015), it was not 
before 2008 that European Parliament and Council issued the EQF recommendations—a ‘translation 
grid’ with eight-level descriptors—and recommended that member states use it as a common reference 
tool to compare qualification levels, relate qualifications systems or NQFs to the EQF and use an learning 
outcome-based approach when defining qualifications. In 2017, the Council (2017) issued slightly 
modified version of these recommendations to address the problematic issue regarding the concept of 
‘competence’, amongst others (cf. Bohlinger, 2019, p. 400). Meanwhile, data from the latest Cedefop 
(2018, pp. 12-13) survey indicated the following: (1) a total of 39 European countries (all 28 member states 
plus another 11 countries) had adopted recommendations and either established or were in the process 
of establishing an NQF, (2) most countries had proposed or adopted the EQF’s eight-level structure 
and established comprehensive frameworks, and (3) 34 countries had formally linked their NQFs to the 
EQF, whilst 29 countries linked theirs to the Bologna framework. Despite the non-binding nature of the 
EQF recommendations, the development of EQF-influenced NQFs in Europe in the last decade has 
been remarkable, notwithstanding the evidence indicating the NQFs'’ inability to fulfil the broader set of 
objectives and purposes they claim (Mikulec, 2017; Bohlinger, 2019). 

Following Jakobi (2009), the main instruments used by the EU when promoting the EQF policy 
transfer and influencing national NQF policy development can be identified. On the level of (a) discursive 
dissemination, the following ideas and goals can be identified: the EQF is supposed to facilitate LLL 
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and employability, promote the mobility and social integration of European citizens and enable greater 
transparency, comparability and portability of qualifications (Cedefop, 2018; Council, 2017). On the 
level of (6) standard setting, as the EU issued the EQF recommendations, it must set clear timelines 
and benchmarks referring to the criteria and procedures for referencing SQF to the EQF, the quality 
assurance principles linked to the learning outcome-based approach, the outcome-based standards and 
the monitoring and credit system principles based on the learning outcome-based approach linked to 
quality assurance and validation of prior learning (Council, 2017). To achieve the EQF policy aims, the 
EU supported candidate countries and member states through (c) financial means, i.e. actions funded 
by the EU programmes (Council, 2017) (in case of Slovenia, the SQF development and implementation 
was supported with European social funds (ESF) in the years 2009-2014 and with Commission grants 
to the EQF national coordination points (NCP) from 2011 onwards). On the level of (d) coordinative 
functions that explore the progress made towards common policy aims, the EQF has been governed 
by the OMC and monitored through the EQF Advisory Group (EQF AG) and other networks (i.e. EQF 
national coordination points and EQF peer learning activities), whilst the implementation of the EQF and 
established NQF is one of the indicators in the two ‘flagship initiatives’ for the Europe 2020 Strategy 
(Mikulec, 2017). Finally, technical assistance is provided in terms of expert advice, because at least two 
international experts should be included in the EQF referencing process (Council, 2017) (in the case of 
Slovenia, three international experts were included). 


Measuring the impacts of NQFs 

Contrary to the numerous publications issued by the EC and contracting agencies in the last 12 
years, which argued in favour of objectives the EQF and NQFs should bring — among others improved 
transparency of national qualification systems, reinforced use of learning outcomes approaches, enabled 
validation of non-formal learning and mobility of learners, enhanced quality of learning and greater 
stakeholders engagement (Cedefop, 2018; cf. Raffe, 2013, p. 147; Werquin, 2007, pp. 466-468; Young, 
2007, pp. 449-500) -, critical scholars emphasised that NQFs are, in fact, unable to fulfil the broader 
set of objectives and purposes they claim and even problematised the idea of measuring the impacts 
of the EQF and NQFs. More than ten years after the EQF implementation, we still ‘know little about its 
actual impact’ (Bohlinger, 2019, p. 403), and member states adopted the EQF ‘on paper while decoupling 
it from Member States’ de-facto implementation’ (Bohlinger, 2019, p. 395). Raffe (2013) pointed out 
that the impacts of NQFs are difficult to investigate, as they are not yet fully implemented and/or not 
functioning for long enough to measure such impacts. Similarly, Allais (2010, 2011) emphasised that the 
NQF'’s impact on the national education and training system may be difficult to measure, because ‘the 
concepts and categories used to measure performance may be changed by the NQF itself’ (Allais, 2010, 
p. 91). She added that the NQF cannot be researched and evaluated as ‘policy mechanisms] in their 
own right’ (Allais, 2017, p. 769), but should rather be examined in a broader context of (education and 
labour market) systems and institutions, the relationship between them and the NQF and other ongoing 
reforms in the country (Allais, 2017, p. 775). Pilcher, Fernie and Smith (2017, p. 9) critically reminded that 
studying the impacts of NQFs is almost impossible due to the semantic vagueness of the term ‘impact’, 
the differences in NQFs’ philosophies and objectives and the methodological complexities; thus, such a 
task is ‘a dream for which it is impossible to identify a suitable yardstick to measure’. Yet, a clear definition 
of the term ‘impact’ and the conduct of independent research focusing on fundamental questions can help 
raise the value of such research. 

The current study’ research design takes these methodological limitations into consideration. The 
SQF’s overall impact is studied 10 years after it came to existence, eight years after its referencing to the 
EQF and five years after the SQF Act was adopted. Moreover, the study is limited to the SQF’s impact on 
the education system (excluding the labour market), by exploring the extent to which SQF objectives have 
influenced changes in education system: (1) improved recognisability and understanding of qualifications, 
(2) improved transparency of qualifications and qualifications subsystems, and (3) reinforced LLL support. 
The study examined the SQF in the context of wider education reforms undertaken before the SQF 
development as well. Finally, the impact is examined from the perspective of those stakeholders, whose 
occupational positions make them the primary users or implementers of the framework: they are the ones 
expected to implement it into various official and curricular documents and use it for different types of 
Career, education and learning guidance and counselling activities. In that regard, they can be considered 
a representative sample. 
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The case of the SQF 


Context and settings 

After gaining independence in 1991, Slovenia reformed its entire educational system in the first half 
of 1990s based on European values regarding the principles of human rights and justice, the autonomy of 
students and teachers, quality provision and comparable European educational standards. Furthermore, 
it introduced the outcome orientation of the curricula and standards of knowledge as the basis for national 
external testing of knowledge, the essential purposes of which were to monitor the quality of educational 
institutions and to strengthen the autonomy of teachers (Ermenc, 2014; Mikulec and Ermenc, 2016). 

In 1998, Slovenia started to prepare for its formal accession into the EU and later became a 
member in 2004. This triggered a second wave of educational reforms, especially in vocational, higher 
and adult education subsystems, which can also be attributed to the extensive financial support from 
European project funds. Slovenia was one of the 29 countries that signed the Bologna Declaration in 
1999 and, in the coming decade, reformed its higher education system in line with Bologna requirements 
leading to the following: (a) a three-cycle degree structure, (b) the use of the European Credit Transfer 
and Accumulation System (ECTS) and learning outcome-based approach in curriculum development, (c) 
the use of the European Standards and Guidelines for Quality Assurance (ESG), (d) the establishment 
of the NQF in line with the EQF and framework of qualifications for the European higher education area 
(FQ EHEA) and (e) the establishment of recognition procedures in line with the Lisbon Recognition 
Convention (Ermenc and Mikulec, 2020). Furthermore, with the Lisbon Strategy set in 2000, Slovenia 
participated in its implementation and fulfilled the Education and Training 2010 programme in vocational 
education and training (VET) and adult education from 2002 on, in addition to monitoring and reporting 
on already achieved objectives. The reforms focused on (a) quality assurance, (b) transparency of 
(vocational) qualifications based on learning outcomes, (c) implementation of decentralised competence- 
based curricula, (d) development of flexible and individualised learning paths, (e) development of the LLL 
strategy, (f) implementation of key competences and validation of informal learning and (g) implementation 
of European transparency tools (SQF, Credit Transfer System for VET (ECVET), Europass) (Ermenc, 
2014). 

Therefore, the implementation of the Bologna and Lisbon processes in the first decade of the 21° 
century triggered a variety of educational reforms, with which the economic goals of education came 
to the fore, and the SQF’s development, which started in 2009 and was one of the last initiatives to be 
implemented from the Lisbon period in Slovenia and built on previously undertaken reforms. 


SQF features 

The SQF development in Slovenia started in 2009, when the ‘Slovenian Qualifications Framework’ 
project was lunched and other supporting bodies were set, including the EQF national coordination point, 
the Interdepartmental Working Group established by the Government involving representatives of key 
stakeholders, and the SQF expert group. First, the SQF draft proposal was prepared at the end of 2010, 
and a comprehensive consultation process with stakeholders took place in 2011 and in the beginning 
of 2012. Then, the SQF’s first referencing to the EQF and its self-certification to the FQ EHEA were 
prepared, discussed with the stakeholders, presented to the EQF AG in 2013 and adopted in 2014 by 
the EQF AG. Finally, at the end of 2015, the special SQF Act was formally adopted by the Slovenian 
government (CPI, 2014; ZSOK, 2015). 

The SQF is comprehensive framework, which includes qualifications from all subsystems of initial 
and further education and training. The SQF includes three types of qualifications: (1) formal educational 
qualifications; (2) vocational qualifications that can also be obtained outside the formal education 
system and consist of (a) national vocational qualifications (NVQs) and (b) qualifications obtained under 
continuing vocational training or study programmes for continuing education; and (3) supplementary 
qualifications, |.e. qualifications that supplement an individual’s competences in a specific professional 
field and are tied to the labour market's needs. Following the example of the EQF from 2008, the SQF 
level descriptors are described in terms of knowledge, skills and competences, although its definitions 
are adapted to the existing national system (CPI, 2014; ZSOK, 2015). The framework, which contains 
10 levels, can be categorised as a ‘communication framework’ (Raffe, 2011, pp. 283-284; Raffe, 2013, 
p. 148), as it is mainly based on existing education legislation and established educational practices. 
Its purpose is to improve the existing qualifications system’s transparency, and the form of learning 
outcomes is not prescribed uniformly for all qualifications in different educations subsystems by the SQF 
(Mikulec and Ermenc, 2016, pp. 6-7). However, the SQF is also designed as a tool of reform. This 
is because, for the first time, it introduced new type of qualifications (i.e. supplementary qualifications) 
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into an existing qualifications system and gave those qualifications coming from the labour market (i.e. 
employers) state recognised visibility. Nonetheless, this type of qualifications is not equivalent to the 
educational or vocational qualifications and there is no horizontal progression between supplementary 
qualifications and educational or vocational qualifications. Furthermore, the SQF also recognised new 
vocational qualifications (i.e. qualifications obtained under continuing vocational training programmes or 
study programmes for continuing education), which support LLL opportunities in higher education and 
foster the greater integration of the labour market and education in continuing VET. 

By making a distinction between the SQF’s communication and reform roles, we can better 
understand its main objectives. The first two, I.e. (a) ‘to improve transparency, accessibility and quality of 
qualifications’ and (b) ‘to connect and coordinate Slovenian qualifications subsystems’, are related to the 
SQF’s communication role, whilst the third one, i.e. (c) ‘to support LLL’ and to integrate the labour market 
needs with education and support mobility, are related to the SQF’s reform role (CPI, 2014, p. 30). For the 
purpose of the empirical investigation, we developed the following three categories related to the above- 
mentioned SQF objectives: (1) recognisability and understanding of qualifications, (2) transparency of 
qualifications and coordination of qualifications subsystems and (3) LLL support. 


Materials and Methods 


A quantitative research approach was employed in order to contribute to the better understanding 
of the impact of the EQF-influenced SQF on its education and training system. We examined the 
achievement of the objectives at a common level, that is, at the level of all stakeholders included in the 
survey, and also looked into the extent to which the SQF’s objectives were met from the perspectives of 
those stakeholders who used the SQF at their work compared to those who do not. We also compared 
the perspectives of those stakeholders who were informed about the SQF’s benefits (or assessed that 
the benefits were presented to them) with those who were not informed. The following research questions 
have been designed: 

RQ1. What are the key stakeholders’ views regarding the extent to which the SQF objectives have 
been met? 

RQ2. Are there any differences in the stakeholders’ views on the extent to which the SQF objectives 
have been met depending on their usage of the SQF? 

RQ3. Are there any differences in the stakeholders’ views on the extent to which the SQF objectives 
have been met depending on whether they are familiar with the SQF’s benefits? 


Survey sample 

In the first phase, all relevant institutions were identified and requested to participate: including 
ministries responsible for education, higher education, labour market and economy; education institutions 
(vocational schools and colleagues, secondary general schools and universities); adult education 
colleges and institutes; students’ organisations and employment service. The institutions were asked to 
identify those employees who best fit the description of the key stakeholder, so that the survey sample 
can be the best expert sample for the study. In other words, we were interested in those who were 
supposed to implement and use the SQF for professional (not personal) reasons, but were who not 
involved in the establishment or design of the framework. There were 50 key stakeholders included in 
the survey. Although the sample is of a medium size, it is representative for the studied population. The 
largest number of the sample included faculty representatives (44%), followed by educational institution 
representatives (18%) and career advisers (12%). The remaining categories—representatives of student 
organisations (4,0%), ministries (2,0%), adult education (4,0%) and vocational colleges (4,0%)—were 
represented to a much lesser extent. 


Instrument 

A questionnaire was designed to collect the data from the identified stakeholders. It consisted of 
three sets of questions, which measured the opinions of the stakeholders on (1) the recognisability and 
understanding of qualifications (5 items), (2) contribution of SQF to the transparency of qualifications and 
coordination of qualifications subsystems (6 items) and (3) the LLL support of SQF (4 items), respectively. 
The items were originally measured on a 5-point scale ranging from ‘disagree’ (grade 1) to ‘fully agree’ 
(grade 5). However, due to the small survey sample size, the grades were merged into a 3-point scale 
wherein grade 1 indicated disagreement, grade 2 indicated undecided respondents and grade 3 indicated 
their agreement. 
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In addition, two additional dichotomous questions, which considered the use of the SQF at work 
and being informed about the SQF’s benefits, were asked. The respondents were given the opportunity 
to answer with ‘I do not know’ at each item, in case they did not have enough information to express their 
opinion. 


Procedure 

The study was conducted using an online questionnaire administered in February 2019. The 
collected data were analysed using quantitative analysis methods, such as basic descriptive statistics 
(frequency distribution). The existence of statistically significant differences regarding the SQF’s benefits 
between users and non-users of SQF and between informed and non-informed stakeholders were 
examined with the Likelihood ratio test. The respondents who provided the ‘I do not know’ answer were 
omitted from the analysis. 


Results 


Recognisability and understanding of qualifications 


Table 1. 

Frequency distributions for items measuring recognisability and understanding of qualifications 
with the results of the likelihood ratio test regarding the use of the SQF and being informed about the 
SQF’s benefits 


Likelihood ratio test 
n Disagree iesia anes ikelihood ratio tes 


LR di p 
Familiarity withthe SQF 0 0 120% 80.0% ee 
Users 30 0.0% 3.3% 96.7% 
14608 ? 0.001 
eee eee Non-users 15 27.0% 16 
Informed about the SQF's benefits i1/ 0.0% 0.0% 100.0% 8.090 2 0018 
Not informed about the SQF's benewits 30 13.3% 13.3% 13.3% 
Awareness ofthe purposes ofthe SQF = 49 16.3% 20.4% re 
Users 30 6.1% 20.0% 13.3% 
4323 2 0.4115 
sc NUR ERL ROR IAC Reta RRa Rae Nonuses 18 200% 220m OM 
Informed about the SQF's benejits i1/ 0.0% 11.8% oo 2 YD 9 5B) 9 0014 
Not informed about the SQF's benefits 30 20.0% 26.1% 543% 
Availability of the materials and other SQF- 
‘elated informationneeded == aa tanreE pte Hetana mag m zi Rie MRE NARA AATEN 
Users 30 3.3% 20.0% 16.7% 
93/0 2 0.009 
apr cna gees Non-users 15 26.7% 400% 
eOOIES about the SQF's benefits i/ 0.0% 0.0% 100.0% 030822 0.000 
Not informed about the SQF's benefits 9 20.7% 41 4% 3f 9% 
Familiarity withthe EQF A 2 Lt, ae Le or mem me moter monsoon: 
Users 30 3.3% 16.7% 60.0% 
6.082 2 0.048 
eee ena eee Non-users 16 20.0% 20.0% 00 eee 
Informed about the SQF's benefits 1/ 0.0% 11.8% Oe YD 6 899 , 0.033 
Not informed about the SQF's benetits 9 1f 2% 24.1% 56.6% 
Contribution of the SOF to a better 
_Understanding of qualifications : eye . ia eer —— Serene a = geo aged nn gape 
Users 27 3.0% 25 9% 10.4% 
22if 2 0.320 
regia eonee pmntnnmnee mee ae emo: PORN NR AB Re ls EERO So RN IM ot wo 
Inf d about the SQF's benefits 15 0.0% 0.0% 100.0% 
pleat about the SQF's benefits 193922 0.000 
Not informed about the SQF's benefits ?/ ft 4% 51.9% 40.7% 


Table 1 shows that the recognisability and understanding of qualifications amongst the stakeholders 
is highest in terms of familiarity with the SQF, which Is typical for SQF users (96, 7%) and for those who 
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are informed about the SQF’s benefits (100%). Familiarity with the EQF amongst the stakeholders is 
lower than familiarity with the SQF, and this finding holds for those who use the SQF at their work and 
those who are informed about the benefits of the SQF. 

About two-thirds of the stakeholders state that they are familiar with the SQF’s purpose, and a 
slightly smaller proportion state that they have access to the materials and other information on the SQF 
that they need. 

Being informed of the SQF’s benefits has a significant impact on the recognisability and 
understanding of qualifications amongst the stakeholders. This is because the levels of familiarity with 
the SQF and EQF, awareness of the SQF’s purpose and the availability of materials and information 
on the SQF are all statistically significantly higher amongst the stakeholders who are informed about 
the SQF’s benefits. Furthermore, the stakeholders who are informed about the SQF’s benefits believe 
that it contributes to better understanding the qualifications compared to the non-informed stakeholders. 
To some extent, the use of the SQF is also a predictor of achieving the goals of recognisability and 
understanding of qualifications, as the levels of familiarity with the SQF and EQF and the accessibility of 
materials and information on the SQF are statistically significantly higher amongst SQF users compared 
to non-users. 


Transparency of qualifications and coordination of the qualifications subsystems 


Table 2. 

Frequency distributions for items measuring transparency of qualifications and coordination of the 

qualifications subsystems with the results of the likelihood ratio test regarding the use of the SQF and 
being informed about the benefits of the SQF 

ri Disagree Undecided Agree Se 

Contribution of tha SOF to a simpler 


description of qualifications = Eyl A sagan nega OT curr repeals 
Users 27 0.0% a. 66.75 Z % 
Nomusers 15 6.7% 53.3% aa oO! 2 Bie 
Informed aboutthe SOF's benefits 15 0.0% 6.7% 93.35 14540 > p04 
Not informed about the SOF's benefits 27 3.7% tb 70% ; 
Contribution of ths QF to greater 
transparancy of knowledge, skills and 42 T.1% 3a. 48 20.5% 
competences ofeach indinadual qualification, ee 
Users 27 0.0% 25.9% 41S 
10.319 2 0.006 
Noreusers 15 20.0 aT 33.3% - 
Informed aboutthe SOF's benefits 715 0.0% 6.7% 93.35% 43.285 > nod 
Not infoemed about the SOF's benefits 27 11.1% 45.15 407 Su - 
Contribution of tha FOF to greater 
transparancy of qualifications in a particular 
fiald of 40 3.0% 45.0% 30.0% 
expertise (a9 construction, agricultura, 
U8 Ae i et PIR PE Oc PEPE TEC a 
Users 25 0.0% 48.05% 22.05 
4713 Z 0.128 
Noruses 15 13.5% 40.0% 457% oF 
Informed aboutthe SOF's benefits 14 0.0% 7.1% 92 9%, 42478 : n.000 
Not infoemed about the SOF's benefits 26 TG 65.4% 25.95 7 
Contribution of tha OF to better 
understanding of tha relations between 40 T.% 47 5% 20.0% 
_ individual types of qualifications eee 
Users 26 3.8% 8.35% oF oh 
2443 2 0.204 
Noruses {4 14.35% 50.0% 5 : 
Informed aboutthe SOF's benefits 14 0.0% 14.3% 85.7 9p 49 560 > npoe 
Not informed about the SQF's benefits 26 11.3% oF oe 30.35 j 7 
cimplitying the understanding of tha 
transition between qualifications acquired 
through fonmal and non-formal education “= cai a a es 
and supplementary qualifications ofthe ser ee 
Users 27 3.7% 25.95% 70.450 
T wl "h 
Noruses 11 27.5% 45.5% yay ee TS 
Informed aboutthe SOF's benefits 715 0.0% 20.0% o0.05 7470 > one 
Not informed about the SOF's benefits 23 17.4% 39.15% 43.550 5 
Facilitating comparison of qualifications for 
the needa of study andwork ype gaps pape pcp 
Uses 27 0.0% 11.1% 30.95. 
10.862 2 0.004 
Noreusers 13 2.1% FS 45 29 . 
Informed about the SOF's benefits 15 0.0% 0.0% 100.0% +1336 > n008 
Not infcemed about the SOF's benefits 25 12.0% 28.0% 6.0% : : 
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In the context of the SQF’s contribution to the transparency of qualifications and the coordination 
of the qualifications subsystems (Table 2), stakeholders assessed the facilitation of the comparison 
of qualifications for the needs of study and work as the highest (75,0%). The results show that there 
is a significant proportion of stakeholders who remain undecided about the SQF’s contribution to the 
transparency of qualifications or do not have enough information about it. Despite the small proportion of 
those who consider the transparency of qualifications to be inadequate based on the following aspects 
and the high shares of those who acknowledge the SQF’s contribution to the transparency of qualifications 
in terms of the following: (1) a greater transparency of knowledge, skills and competencies of each 
qualification (59,5%); (2) a simpler description of qualifications (57,1%); (3) simplified understanding of 
the transition between qualifications acquired through formal and non-formal education and the SQF’s 
supplementary qualifications (57,9%), (4) contribution to a better understanding of the relations between 
individual types of qualifications (50,0%); and (5) contribution to the greater transparency of qualifications 
in a particular field of expertise (50,0%). 

The stakeholders’ opinions regarding the SQF’s contribution to the transparency of qualifications 
and to the coordination of qualifications subsystems are significantly dependent on whether or not they 
are informed about the SQF’s benefits. The stakeholders who are informed about the SQF’s benefits 
recognise its contribution to the transparency of qualifications and to the coordination of the qualifications 
subsystems in all respects to a statistically significantly greater extent than those who are not informed. 
The latter are most often undecided regarding the contribution, whilst those who are informed about the 
SQF’s benefits have a mostly high level of agreement regarding the SQF’s contribution to the transparency 
of qualifications and to the coordination of the qualifications subsystems. 

Furthermore, the usage of the SQF conditions the respondents’ views regarding the SQF’s 
contribution to the transparency of qualifications and the coordination of qualifications subsystems to some 
extent. Notably, the SQF’s contributions to the achievement of greater transparency of knowledge, skills 
and competences of individual qualifications; to the simplification of the understanding of the transition 
between qualifications acquired through formal and non-formal education and the SQF’s supplementary 
qualifications; and to a comparison of qualifications for the needs of study and work, are all recognised to 
a Statistically significantly greater extent by SQF users compared to non-users. 
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Lifelong learning support 


Table 3. 
Frequency distributions for items measuring lifelong learning support with the results of the likelihood 
ratio test regarding the use of the SQF and being informed about the benefits of the SQF 


Likelinood ratio test 
Di Undecided A 
n isagree ndecide gree IR tf i 


Contribution to a better understanding and 
comparability of Slovenian qualifications 39 12.8% 20.5% 66.7% 


ee 
Users 27 T.4% 22 2 70.4% 
ee Non-users 12 250% 167% 583% 
1 0 
ee about the SQF’s benefits 15 0.0% 0.0% 100.0% 16.544 > 0.000 
Not informed about the SQF's benejits 24 20.8% 34.3% 45 3% 


Contribution of the SQF to a more 
successful dialogue between the labour 
market and education and training 


institutions, eee. 
Users 24 4 2% 45 8% 50.0% 
5 0o-4 2 0.011 
Se ie Non-users 12 0 40f% AN 16 
Informed about the SQF's benefits 74 0.0% 4? 9% 5f 1% 7999 ; 0.020 
Not informed about the SQF’'s benefits 27 2 3% 45 5% Zt 3% 
Contribution of the SQF to a better match 
between available market Knowledge and 395 14.3% 04.3% 31.4% 
_skills,andworkplacenmeed ee 
0 
Users 24 6.3% 4 3f 5% 2714 , 0.957 
isi icnten ncn Shs aria nets BRON adt Mba rescsne ht A a tesa rare Soa pa arr OEE Pasa tems anemia ma Rem RIG 
Informed about the SQF's benefits 14 0.0% 4? 9% 5f.1% 10.521 9 0.005 
Not informed about the SQF's benefits 1 24.0% 61.9% 14.3% 
Facilitating easier planning of workers’ 
_needs and their professional development __ : ? rea ron apirenares s ae rugs - : i MMP 
Users 23 13.0% 43.5% 435% 
4510 2 0.105 
oe ee Non-users 12 0 407% AN 16 
| tt 
micenes about the SQF’s benefits 13 Tot 15.4% 76.9% 17.558 ; 0.000 
Not informed about the SQF’'s benetits 27 31.8% 59 1% 91% 


The SQF’s LLL support seems to be the weakest area amongst the ones studied (Table 3), as only 
about one third of the sample acknowledge the SQF’s contributions to a better match between available 
market knowledge and skills and workplace needs as well as to easier planning of workers’ needs and 
their professional development. 

The SQF’s contribution to a more successful dialogue between the labour market and education 
and training institutions is acknowledged by more than one third of the stakeholders (38,9 %), with those 
using the SQF at their work assessing this contribution to a statistically significantly higher extent than non- 
users. Furthermore, the stakeholders who are informed about the SQF’s benefits statistically significantly 
differ in their assessment of the contribution from those who are not informed about such benefits. Two 
thirds of the stakeholders believe that the SQF contributes to a better understanding and comparability of 
Slovenian qualifications abroad. Moreover, from all measured aspects, the stakeholders who are informed 
about the SQF’s benefits recognise its LLL support to a statistically significantly greater extent than those 
who are not aware of the SQF’s benefits. 


Discussion 


The EQF policy transfer influenced SQF development through discursive dissemination, |.e. 
common objectives related to facilitation of LLL, such as the promotion of mobility, transparency and 
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comparability of qualifications; standard setting, |.e. clear timelines and benchmarks related to the use of 
criteria and procedures for referencing the SQF to the EQF; financial means, i.e. the SQF development 
and implementation being supported with ESF and Commission grants to the EQF NCP; coordinative 
functions, |.e. progress made towards common policy aims (the SQF) being monitored through the EQF 
networks (EQF AG and EQF NCP); and through technical assistance, i.e. three international experts were 
included in the EQF referencing process. The SQF was built on two waves of reforms undertaken before 
SQF development: one related to the comprehensive education reform in 1990s and another which made 
Slovenia part of the Europeanisation process. The SQF’s development started in 2009 and was formally 
established based on these two waves of educational reforms, which provide the necessary context for 
better SQF analysis and understanding (cf. Allais, 2017). 

As Is the case with most EQF-influenced NQFs in Europe (cf. Cedefop, 2018, p. 17), the SQF 
has mostly been designed as a communication framework (Raffe, 2013), which includes some reform 
elements related to the support for LLL, namely, the mobility and integration of labour market needs with 
education (CPI, 2014, p. 29). 

The results indicate that the SQF is more successful in its communication role compared to 
the reform role: objectives related to the SQF’s communication role are, from the key stakeholders’ 
perspectives, realised to a much greater extent compared to those related to the SQF reform role. Hence, 
objectives related to the reform role, except for mobility, seem to be a much more difficult nut to crack. 

The finding that those stakeholders who use the SQF in their work and/or are informed about its 
benefits assess the framework’s communication role to be much better, compared to those who do not 
use it or are not informed about its benefits, implies that the communication role is a productive one that 
may be even improved in time when more people are informed and eventually become SQF users. This is 
particularly true where recognisability and understanding of the SQF is concerned, but to a lesser extent 
when transparency and coordination of qualifications subsystems are involved. As for the latter, the high 
proportion of the undecided users is still rather high, although this improves in those who are informed 
about the SQF’s benefits. 

More specifically, the majority of the stakeholders agree that the SQF contributes to the recognisability 
and understanding of qualifications; two thirds also agree that it improves the EQF’s recognisability. 
More than half agree that the SQF improves the transparency of qualifications and coordination of the 
qualifications subsystems according. This finding is in line with Cedefop’s (2018, p. 16) conclusions on 
the early impact of the European NQFs. We additionally found that the stakeholders who are informed 
about the benefits of the SQF, along with those who use the SQF (to slightly lesser extent), recognise 
the contribution of the SQF to the transparency of qualifications and coordination of the qualifications 
sub-systems to a statistically significantly greater extent compared to those who do not use it or are not 
informed about its benefits well enough. This finding implies that the NQFs’ role in the transparency of 
qualifications and coordination of the qualifications subsystems may improve after more people who use 
it professionally gain a deeper understanding of the framework’s roles and benefits. 

When it comes to the SQF’s support to the LLL, a disparity of results can be noticed. Whilst two 
thirds of the stakeholders believe that the SQF contributes to the mobility of learners and workers on 
the one hand, less than 40% of the respondents acknowledge the SQFs contribution to other objectives 
related to the LLL policy on the other hand. For instance, only 38.9% stakeholders agree that the SQF 
can contribute to a better integration of labour market needs and education despite the fact that the SQF 
introduced supplementary qualifications that are tied to the labour market needs into the SQF. Such a 
finding may be influenced by the sample’s composition sample (the labour market representatives were 
excluded), yet a recent CPI (2020) report, which included a broader range of respondents, revealed that 
supplementary qualifications were not successfully implemented, because they were not ‘organically’ 
integrated into the existing education system and thus functioned as an alien body causing tensions 
amongst different stakeholders and decision-makers. The SQF’s reform role has not proven to be 
productive in terms of including new types of qualifications, such as those that did not exist before the 
framework was introduced, into the SQF. It seems that the disparity can also be explained along the line 
of the SQF’s communication-reform functions: mobility has been part of the Slovenian education policy 
ever since Slovenia became a member of the EU in 2004 and is well-embedded in the education practice 
with full infrastructure in place. The SQF is merely one element in the mobility policy, thereby enhancing 
but not enabling it. In comparison, the supplementary qualifications are not part of any background policy 
or pre-existing practices. 
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Conclusion 


The indirect measurement of the impacts of the SQF and the small sample prevented us from 
drawing very solid conclusions. Therefore, the findings of the study should be seen as an addition to the 
previous studies investigating the influence of the European education policy on national educational 
policies, focusing on the development and implementation of NQFs in Europe. The study addresses the 
tensions and growing gap between the advocates of the EQF and NQFs. Such a gap arises from the IOs 
and their belief that NQFs and EQFs are a ‘panacea’ that can resolve many educational problems faced 
by Europe on the one hand, and the critical research community arguing that the EQF and NQFs are 
unable to fulfil a broader set of objectives and purposes as they claim on the other hand. By examining the 
case of the Slovenian framework—an example of an EQF-influenced NQF—from the key stakeholders’ 
perspectives, we tried to find a productive way of approaching the impact measurement problem. A more 
balanced and larger sample would enable us to reach less tentative conclusion, yet the results persuade 
us to agree with those who claim that the NQFs indeed cannot function as a panacea. They do, however, 
seem to have the potential to become ‘the cherry on top of the cake’ of more comprehensive reforms 
if implemented as a final policy step supporting other relevant policies related to the recognisability, 
transparency, mobility of qualifications, system coherency and education—labour market cooperation. 
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